feat: Wire LDAIMetrics tool_calls and duration_ms into tracker#152
Closed
jsonbailey wants to merge 4 commits intojb/aic-2388/managed-graph-resultfrom
Closed
feat: Wire LDAIMetrics tool_calls and duration_ms into tracker#152jsonbailey wants to merge 4 commits intojb/aic-2388/managed-graph-resultfrom
jsonbailey wants to merge 4 commits intojb/aic-2388/managed-graph-resultfrom
Conversation
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
98e3ccc to
cbe2eda
Compare
jsonbailey
changed the title
This was referenced Apr 28, 2026
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
d847d83 to
9c9e802
Compare
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
cbe2eda to
4f12618
Compare
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
9c9e802 to
5fb879e
Compare
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
4f12618 to
351f249
Compare
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
5fb879e to
4319954
Compare
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
351f249 to
ef03235
Compare
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
4319954 to
ba64a62
Compare
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
ef03235 to
e5ecc6a
Compare
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
ba64a62 to
49e95a3
Compare
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
e5ecc6a to
1636446
Compare
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
49e95a3 to
80d4e11
Compare
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
1636446 to
0fae2df
Compare
…nnerResult - OpenAIModelRunner.run() implements the unified Runner protocol; returns RunnerResult with content, metrics (LDAIMetrics), raw, and parsed fields. Structured output is supported via the output_type parameter. - OpenAIAgentRunner.run() updated to return RunnerResult; populates tool_calls in LDAIMetrics from observed openai-agents ToolCallItems. - Legacy invoke_model() and invoke_structured_model() retained as deprecated adapters that delegate to run() and wrap results into ModelResponse / StructuredResponse for backward compatibility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… RunnerResult - LangChainModelRunner.run() implements the unified Runner protocol; returns RunnerResult with content, metrics (LDAIMetrics), raw, and parsed fields. Structured output is supported via the output_type parameter. - LangChainAgentRunner.run() updated to return RunnerResult; populates tool_calls in LDAIMetrics from observed tool_calls in message responses. - Legacy invoke_model() and invoke_structured_model() retained as deprecated adapters that delegate to run() and wrap results into ModelResponse / StructuredResponse for backward compatibility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rResult types - Add GraphMetrics dataclass (runner-layer return type for graph runs) - Add GraphMetricSummary dataclass (managed-layer metrics, analogous to LDAIMetricSummary for single-model invocations) - Add ManagedGraphResult dataclass (managed-layer return type from ManagedAgentGraph) - Add AgentGraphRunnerResult dataclass (future runner return type, no evaluations field) - ManagedAgentGraph.run() now returns ManagedGraphResult with GraphMetricSummary built from the runner's AgentGraphResult metrics - Export all new types from ldai package Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
80d4e11 to
05e1f4c
Compare
jsonbailey
force-pushed
the
jb/aic-2388/enrich-metrics
branch
from
0fae2df to
a468a7f
Compare
jsonbailey
force-pushed
the
jb/aic-2388/managed-graph-result
branch
from
05e1f4c to
0a1ee15
Compare
Contributor
Author
|
Closing in favour of the cherry-pick in #148 — the tracker test coverage was moved into the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pins down the contract for the new
tool_callsandduration_msfields onLDAIMetrics. The integration was introduced incrementally as the OpenAI and LangChain runners migrated; this PR adds test coverage that will guard the contract going forward.LDAIMetrics.to_dict()exposestoolCallsanddurationMstrack_metrics_of/track_metrics_of_asyncuse a runner-suppliedduration_msdirectly when present, otherwise measure elapsed wall-clock timetrack_metrics_ofcallstrack_tool_calls()whenmetrics.tool_callsis present, emitting one$ld:ai:tool_callevent per key and recording the list onLDAIMetricSummaryStack
PR 7 → PR 8 → PR 8-openai → PR 8-langchain → PR 9 → PR 10 (this) → PR 11 → PR 11-openai → PR 11-langchain → PR 12
Test plan
make test— 267 tests passmake lint— mypy clean🤖 Generated with Claude Code